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placed in the cart, (3) it prompts the customer to make payment and updates 
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and a few direct current motors programmed to achieve autonomy. We used 
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Database an open-source cross platform software called XAMPP to create the database 
Raspberry Pi and used RFID tags to bill the items placed in the cart automatically. The 
RFID system updates payments and communicates these transactions to a local 
Supermarket database via nRF24 wireless transceivers. The experimental tests conducted 


demonstrate that our system successfully followed customers accurately 
within the mall. We consider our design a major contribution to the vision of 
automated shopping systems for the near future. 
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1. INTRODUCTION 

In modern shopping centres (malls and super markets), shopping carts are often seen as plastic or 
metal basket-like frames on wheels pushed by customers to carry items or products around the centre [1], [2]. 
However, shopping via traditional shopping carts in modern centres may pose several challenges to 
customers including the burden of pushing heavy carts by the elderly, physically challenged, pregnant and 
nursing mothers. Other salient challenges may include delay at long queues, prolonged sorting and billing 
details at the till. Furthermore, mall managers often need to track manually each stock available and sold in 
the large centre, which can be burdensome. These challenges require innovative solutions, which served to 
motivate the ideas demonstrated in this paper. 

Several authors have proposed different approaches to address the above challenges in the literature. 
However, these efforts focused more on autonomous mobility of the shopping cart based on the 
line-following approach [3]-[7]. Image processing has found application in many fields among which include 
[8], [9]. Its used for autonomous mobility of shopping cart based on some image colours for tracking object 
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as well as some android application-controlled approaches were presented in [10]-[14]. While, other efforts 
addressed automated billing system issues and developed predictive algorithms to suggests items to 
customers [15]-[17]. A multifunctional automated trolley system based on an android application controlled 
via a smart phone fixed to a trolley was presented in [2]. Their system achieved good performance. However, 
it required a predefined black marked line in order to guide the cart’s movement. These predefined markers 
typically constrain the customer’s movement thus limiting the flexibility of their design. Similarly, authors in 
[3] used a line-following robot to drive a shopping cart. They deployed an ultrasonic and line sensor to track 
the customer’s movement and an android application to guide the customer to where goods were located in 
the mall. They used also an RFID localization scheme and an android application to track the trolley’s 
location. Nevertheless, the line following approach in [3] limits the flexibility of their design similar to [2]. 

Authors in [4]-[6] used ultrasonic sensors to detect obstacles and to track customers. They used 
Arduino integrated development environment (IDE) to drive a system of motors to provide autonomous 
mobility to the cart. Their use of ultrasonic sensors may be similar to other use-cases found in [18], [19]. 
However, their designs in [4]-[6] are typically unable to differentiate actual users from other stray carts that 
infiltrate the range of the ultrasonic sensor. A mobile following robot capable of avoiding obstacles, while 
tracking user was developed in [7]. By using preprogramed images of different objects, authors demonstrated 
that their system could successfully track customers. A similar concept was presented in [20] that could 
navigate and avoid obstacles. Authors used this concept to develop autonomous shopping carts. Authors in 
[21] developed an algorithm to search and map the position of a cart user. They used laser range sensors to 
estimate a user’s position and distance, a gyro-sensor and a rotary encoder. However, their system’s accuracy 
depends strongly on the mapping algorithm, which can be often slow due to complex computations. 

Kim et al. [22] investigated different human perspectives toward the design of a convenient and 
efficient smart cart system. They used a questionnaire and survey method to investigate the best position of 
an RFID tag and reader, touch screen, a barcode system and a cart module in the smart cart system. The 
results obtained showed that most customers prefered the touchscreen to be located at about 250 mm from 
the handle of the cart, while the cart handle should be tilted at 45 degrees. A similar feasibility study was 
conducted in [23] to develop an intelligent cart system for use in airports. Authors used the ant colony 
clustering (ACC) algorithm to find quasi-optimal positions to assemble the cart. The waiting time and precise 
route of the carts were determined via simulation and detailed procedures were used to compute a sequence 
of control commands to move the cart. Authors provided results to show that their approach aligned and 
arranged scattered carts automatically. Marchetti et al. [15], a data fusion algorithm based on particle filter 
was developed to combine information from an RGB-D camera and a planer laser range finder to estimate 
and detect the position of a user. These were incorporated into a proposed autonomous shopping cart to track 
users. To estimate the user’s position, authors in [15] adopted a k-clustering technique. They demonstrated 
that their system consistently tracked the users. Prerana et al. [16] developed an approach that used an RFID 
reader, which automatically scanned and billed products dropped in the cart. The system added the cost of an 
item automatically to a list of billed items once the RFID reader scanned it. The customer would activate a 
switch button to detect the product code in order to remove items no longer required. The system transfers 
the final product bill via zigbee to the counter where payment can be made. Despite demonstrating the 
capability of autonomous billing, the system in [16] fails to incorporate autonomous mobility and automatic 
payment systems in their model. 

The approach in [17] used machine learning technique to enable a robot to learn and modify its 
knowledge about different customer shopping preferences. The system logged a repository of digital data 
about the user’s cultural preferences into a database. It used this information to profile each user and to 
predict each user’s preference. In their design, an inbuilt camera captures images used to train the system to 
identify the user’s emotional status. The results obtained demonstrated the feasibility of their approach. 
However, their approach is relatively complex because it requires large processing capabilities, which makes 
it costly for real time realization and implementation. 

Inspired by past efforts, we provide in this paper an encompassing design of an automated shopping 
cart system that improves flexibility, provides instantaneous billing and automated inventory tracking via a 
local database. The proposed new automated smart cart system used a simple combination of an ultrasonic 
sensor, Raspberry Pi, some direct current (dc) motors, a video camera, RFID tags and readers, an nRF24 
wireless transceiver, an LCD display, a GSM module and a database. Our design provides a low cost 
innovative smart shopping cart that tracks accurately the customer, provides automatic billing, and displays 
each unit cost and the total cost of each item on an LCD screen. Furthermore, our system keeps track of all 
inventories for shop managers/owners. We improved on the approach proposed in [5], [6] by using ultrasonic 
sensors instead of the line-following technique. We improved also the billing system over the approach in 
[16] by providing a notification system along with using a local database to keep track and update of stocks 
in the mall for the shop owner. In addition, we conducted a detailed analysis to examine the performance of 
our system and its suitability for commercial production. The remainder of the paper is structured as shown 
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in: section 2 presents the research method of the proposed smart innovative shopping cart system. Section 3 
presents the experimental results and discussion while section 4 concludes the paper. 


2. RESEARCH METHOD 

We adopted a three-stage methodology in our design namely, (1) to develop the autonomous smart 
cart system (2) to build the billing system and (3) to design the database modules. A schematic of our design 
is shown in Figure 1 and we explain each stage as: 


2.1. Autonomous smart cart development 

The autonomous smart cart consists of a reinforced aluminium sheet frame supported on a two 
wheel and a rear-supporting wheel. It contains a basket-like opening, where customers could drop items 
while shopping. A central processing unit (Raspberry Pi) coordinates and controls the movement of the cart. 
To provide autonomous mobility to the cart, we adopted an image-processing algorithm based on the RGB 
colour-processing model [24]. The algorithm acquires an object frame via a camera installed in the top front 
of the smart cart system, which makes up the video acquisition unit as shown in Figure 1. Similar to the 
threshold computation approach presented in [25], the algorithm compares each newly acquired object frame 
with pre-saved object frames in the system and then computes a threshold pixel level using (1). 


D = XJ (R2 — R1)? + (G2 — G1)? + (B2 — By)?” (1) 


where, R,,G,&B, are the current object image pixel acquired, while R,,G,&B are the pre-saved 


background object image pixel. If the computed threshold level is less than 500pixel value, the system 
assumes that the user is found. Else, if the computed pixel value is higher, then the user is assumed not to be 
within the range of detection by the smart cart. The system adjusts always the location of the pixels computed 
after detection to the centre of the camera. It uses this adjustment to direct the cart to go either left or right for 
the horizontal axis and to go either forward or standby for the vertical axis. This guarantees that the user is 
continuously tracked. 

The system acquires two-dimensional object images (x, y) through the camera to determine 
continuously the position of the user. We used an object-image window size of 320 by 240 pixels to 
determine the user’s position. Usually, if the computed horizontal axis (x-coordinate) of the image is less 
than 140 pixels, the cart detects the user to be at the right, but if it is greater than 200 pixels, then the cart 
detects the user to be at the left side due to a 180-degree image inversion of the camera. Similarly, if the 
computed vertical axis (y-coordinate) of the object image is greater than 170 pixels, the cart assumes that the 
user is a little farther away and triggers the motor incorporated in the driver unit to move the cart. Else, if 
otherwise, the cart remains in the standby mode. We provide a summary of this image-processing algorithm 
in Algorithm 1. The driver unit comprises of a motor driver integrated circuit (L298N), two 12 volts dc 
motors and a battery. The output of the video acquisition and processing unit trigger the motor driver via the 
Raspberry Pi processor. We labelled the two dc motors as motor 1 and 2, respectively. Each motor had two 
terminals labelled as A and B and controlled by two logic states 1 and 0. The logic 1 signifies a high (on) 
state, while logic O signifies a low (off) state. When motor 1 and 2 received a logic combination of 1 and 0 at 
terminal A and B respectively, it implies that the cart will move in the forward direction. If any of the 
terminals is reversed for each motor, then the cart will move either left or right direction. However, if all the 
terminals are at logic 1 or O for both motors, it implies that the cart will be on standby mode, i.e. in a 
stationary position. We summarize in Table 1 the movement pattern of the proposed cart based on the logic 
received from the Raspberry Pi. Note in Table 1, that N/A denotes that the coordinate values are not 
applicable to the movement pattern of the cart. 

In the obstacle detection unit, an ultrasonic sensor, which is a low-cost transceiver, was utilized. The 
ultrasonic sensor has a separation distance of 2 cm between the transmitter and receiver, which was used to 
approximate the distance between the transmitted ultrasound signals at 40 Hz from the cart. This determined 
the time taken to receive the reflected echo pulse from the obstacle along the path as shown in Figure 2. We 
used an HCSRO4 ultrasonic sensor to measure the distance between ranges of 2-400 cm. If the measured 
distance by the ultrasonic sensor using (2) is less than the set threshold value (20 cm), the Raspberry Pi 
triggers the cart to remain in a standby mode since an obstacle apparently exists in its path. Other details and 
configuration of the principle of operation of the ultrasonic sensor used for the obstacle detection can be seen 
in [18]. We used an open source processing software environment to program the Raspberry Pi. 
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where dis the distance computed in (cm), heis the echo pulse width high time, and vis Sound velocity (m/s). 
The sound velocity is 340m/s or 29microseconds per cm. 





AUTONOMOUS SMART CART Power supply and Distribution BILLING SYSTEM ALGORITHM 1 
SYSTEM 1. Initialise the cart 
Video Acquisition 2. The Microcontroller triggers the video camera. 
ii RFID Reader 3. Microcontroller get the background object data from the 
camera feed. 


4. Microcontroller process the background data and pre- 
save it. 
5. Microcontroller continuously get video frame of the 


Display unit (LCD) current object. 
EN 6. Measure the difference between the current object data 
and the pre-saved data. 


Microcontroller 


(Raspberry PI) 


Obstacle detection 


unit 





7. Is the difference below the set threshold? 


If yes, 
innti a. object found go to step 8 
AATA Notification (Buzzer in ject found g P 
, 


and LED indicator) 
inRF24) b. Object not found go to step 5. 


8. Compute the horizontal position of the object pixel (x) 
a. if the x-coordinate is less than 140, turn right 


DATABASE Else, a 
b. If the x-coordinate is greater than 200, turn left. 


9. Compute the vertical position of the object pixel (y) 
a. if the y-coordinate is less than 170, move 
Figure 1. Block diagram of the proposed system architecture and forward 


: : Else 
implementation 
P b. Remain in standby mode 


10. Return to step 5 











Table 1. Proposed smart cart movement pattern 








x-coordinate -coordinate Direction Motor I Motor 2 
N/A <170 Forward A: 1 B:O <A: 1 B:0 
N/A >170 Standby A: 1 B:1l A: 1 B:1 
<140 N/A Turn Right A: 0 B:1 <A: 1 B:0 
>200 N/A Turn Left A: 1 B:0O A: O B:1 


ALGORITHM 2 
. Initialize the billing system 
. RFID reader automatically scan the item 
. RFID reder fetch the item ID and send 
to microcontroller 
. Microcontroller send the item ID via 
nRF to database 
. Database check for the item validity, 
Is tag id valid? 
a. Yes, fetch the item name and 
price 
b. No, return invalid 
. Display the item name and price on LCD 
. Add the item price to the customer bill 
. Is item removed? 
a. No, goto step 10 
b. Yes, scan the item ID and 
deduct the price from the bill 
9. Update the bill on the LCD 
Figure 2. Circuit diagram of the autonomous cart 10. Print the Bill 


11.Go back to step 2 





2.2. The billing system 

This module comprises of an RFID reader operating at a frequency of 125 kHz, incorporated inside 
the basket-like opening in the smart cart. We tagged each item using a unique ID in all our experiments. This 
enabled the system to scan automatically each item placed in the cart. Usually, the RFID reader would read 
these IDs via the serial pin of the Arduino Uno board. The Arduino board controls the billing module, as 
information received from the transmitter (RFID reader) is sent to the developed database as shown in 2.3. It 
achieves this via the nRF24 wireless transceiver, interfaced to the Arduino using a serial peripheral interface 
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(SPI). We covered the outer part of the smart cart with a radio frequency (RF) shield to prevent the RFID 
reader from scanning other products outside the cart. The system extracts from a database the name and cost 
of each item placed in the cart and displays these on an LCD. As the items placed in the cart increases, the 
cost of each item is added to the cost previously extracted from the database and this is displayed on the 
LCD. When a user changes his/her mind about an item and removes it from the cart, the RFID scans 
automatically the item and communicates its ID to the database, which updates automatically the list of 
intended purchased items and their corresponding cost. We summarize this entire process in Algorithm 2. 


2.3. Database module 

This section describes the local database that contains information about each item stocked in the 
mall. We created the database using the MySQL and the XAMPP control panel, which is an open source 
cross platform. We used the MySQL and Apache from the XAMPP control panel in our design. The domain 
"localhost/phpmyadmin/” was launched via the address bar of a Google Chrome browser. We named the 
database as "shopping cart and table" and displayed this on the phpmyadmin interface, which indicates that 
the database was created successfully. The phpmyadmin interface launches the created "shopping cart and 
table" database. The "Database Shopping Cart" was assigned. Five different fields namely; (1) serial number 
(s/n), (2) tag ID, (3) item name, (4) item price, and (5) amount in stock. The data type for each field was 
assigned and the data format type called “integer” was assigned to S/N and to amount in stock, while the 
variable character is assigned to all other fields. Further details concerning the creation of the database with 
other necessary fields using the XAMPP software can be found in [26]. The database was interfaced with an 
ethernet shield and an Arduino Uno board, which retrieves from the billing module any necessary 
information requested by the item ID as shown in section 2.2. The retrieved information from the local 
database is sent to the billing module via the ethernet shield and the nRF transceiver. The overall system 
architecture comprising of all the three modules is summarised in Figure 3. 


SYSTEM ARCHITECTURE 


AUTONOMOUS CART BILLING UNIT DATABASE 
INITIALISATION 


SCAN ITEMS 
COMPARE FRAME 
2 h ADD ITEMV ALUE 
FRAME TO THE BILL 


INITIALISATION 


LISTEN TO 
MESSAGE 


, NO| PRINT 
> 
ID VALID? INVALID FORWARD ID TO 
DATABASE 


ES 


DEDUCT RE TRIVE THE ITEM 
TTEMREMDV ED? - ITEM IDDETAILS 
VALUE 


SEND ITEMDETAILS TO 
BILLING UNIT 





Figure 3. Overall system architecture methodology flowchart 
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3. RESULTS AND DISCUSSION 

We tested the new smart cart system in two stages. First, we tested separately the individual 
modules of the proposed system viz the autonomous module, the billing and database module. Secondly, we 
tested the communication system between all the three modules as well as the entire system. In testing the 
autonomous module of the proposed system, the camera feed rate was set to 30 frames per seconds, while the 
best orientation of the camera that gives better result was investigated as summarized in Table 2. Observe 
that an average response time of 33ms was required to trigger the autonomous tracking of the user. We 
designed the billing system to scan an item at a time, particularly as each item was dropped in the cart. The 
system had a delay of 500 ms after scanning automatically the item placed inside the cart. This delay was 
necessary in our design to enable the system to communicate with the local database in order to fetch the cost 
of each item, compute and sum the total bill of items in the cart. This was initialized by the pre-saved colour 
of an object that will usually be attached to the user’s body. We measured this response time only when the 
user is within the range of operation of the proposed system, which is in the range of 0 to 475 cm. The system 
becomes non-responsive at ranges greater than 475 cm as shown in Table 2. The system remained in a 
standby mode when the user was positioned within a range of 0—50 cm. In this case, it did not matter whether 
the user was positioned directly in front of the cart or on its sides. In addition, the system performed well in 
terms of successfully tracking the user, especially when the user was positioned directly in front of the cart. 
However, it performed poorly when the user was positioned at the bottom plane as shown in Table 2. 
Generally, an average-response scan time of about 300 ms after 60 trials was recorded for each item as 
shown in Table 3), with a corresponding bill computation time of less than 850 ms at varying distances 
between 1 to 5 cm. Nevertheless, the system may remain non-responsive and thus continue to scan 
indefinitely when the scanning distance is greater than 5cm as shown in Table 3. The screen shot of the 
created database structure and its fields are shown in Figure 4. We provide also in Figure 5 an image of the 
physical prototype of the cart system. 


[D] XAMPP tutoniat create your own x | MA localhost / 127.0.0.1 / shoppinge X | + 


é > Q © localhost/phpmyadmin/sql.php?db=shoppingcart&table=shoppingcart_table&tpos=0 


Ht Apps For quick access, place your bookmarks here on the bookmarks bar. Import bookmarks now.. 
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roe SELECT n 1 
B@cet ih 
H a New 
+} cartable 
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PH | mysg 
P- | performance_schema + Options 
E- phpmyadmin -T> v SN TagiD Item Name item Price Amount in Stock 
—_4 request ) p Edit 3}: Copy © Delete 3 190079594871 Tomato 100 963 
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| Query results operations | 
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Labet: | _ Let every user access this bookmark 








Figure 4. Screenshot of the created database Figure 5. Prototype of the prposed 
innovative shopping cart system 


Table 2. Performance response time and tracking by the proposed system 


Position Range (cm) Response time (ms) Action Performance 
Sides (right/left) 0-50 33 Standby Fair 
50-475 33 Left/right/forward 
>500 No response 
Top (center) 0-50 33 Stand by Very good 
50-450 33 Left/right/forward 
>485 No response 
Bottom plane 0-50 33 Standby Poor 
50-185 33 Left/right/forward 
>200 No response 
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Table 3. Average scan duration and response time of billing module 
Scan distance (cm) Acerage scan duration Average response time (bill computation) 


1 < 300 ms < 850 ms 
2 < 300 ms < 850 ms 
3 < 300 ms < 850 ms 
4 < 300 ms < 850 ms 
5 < 300 ms < 850 ms 
6 Infinite No response 


4. CONCLUSION 

This paper has presented the design and prototyping of an innovative smart shopping cart system. 
The autonomous capability of the system was achieved using pre-saved object images that were processed 
and used to trigger the mobility of the cart. A Raspberry Pi processor was used to coordinate the cart’s 
movement. In addition, a local database was created using XAMPP software, which contained the stocked 
items and their corresponding ID, price and the number of stocks. The database keeps track of available 
stocks in the mall. Furthermore, the billing module of our system communicates via the nRF24 wireless 
transceiver to fetch information required from the database. This was achieved by scanning automatically the 
item’s ID as they were dropped in the cart. Our test results showed that the cart system tracks and follows the 
user successfully and automatically particularly when it was triggered by the pre-saved colour image of an 
object placed on the user’s body. Our system also billed items automatically that were dropped in the cart. It 
similarly updates successfully the stock on the database. Future works will consider how to integrate in our 
design an online payment unit in addition to providing automatic battery charging and cart parking capability, 
this will further improve the efficacy and performance of the system. 
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